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I. REAL PARTY IN INTEREST 

The invention is assigned to Intel Corporation of 2200 Mission College 
Boulevard, Santa Clara, California 95052. 

II. RELATED APPEALS AND INTERFERENCES 

To the best of Appellant's knowledge, there are no appeals or interferences 
related to the present appeal that will directly affect, be directly affected by, or have a 
bearing on the Board's decision. 

III. STATUS OF THE CLAIMS 

Claims 1, 3, 5, 7-13, 15, 17, 19-27, 29-30, 32-33, 35 and 39-42 are 
pending in the above-referenced patent application. Claims 1, 3, 5, 7-13, 15, 17, 19- 
27, 29-30, 32-33, 35 and 39-42 were rejected in the Final Office Action mailed on 
November 24, 2004. Claims 1, 3, 5, 7-13, 15, 17, 19-27, 29-30, 32-33, 35 and 
39-42 are the subject of this appeal. 

IV. STATUS OF AMENDMENTS 

An amendment was filed on August 09, 2004 and was considered by the 
Examiner prior to issuing the Final Office Action. 

A copy of all claims on appeal (claims 1, 3, 5, 7-13, 15, 17, 19-27, 29-30, 
32-33, 35 and 39-42) is attached hereto as Appendix A. 
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V. SUMMARY OF CLAIMED SUBJECT MATTER 



The present application discloses a method and apparatus for a language 
independent, voice-based Internet or intranet search system. Techniques disclosed in 
the present application may be used to enrich the current Internet or intranet search 
framework by allowing users to search for desired information via their own native 
spoken languages. In one embodiment, the search system may accept voice input data 
from a user spoken in a conversational manner, automatically identify the language 
spoken by the user, recognize the speech in the voice input data, and conduct the 
desired search using the speech as input data for a search query to a search engine. 
To make the language independent voice-based search system even more powerful, 
several features may also be included in the system. Natural language processing 
(NLP) may be applied to extract the search terms from the naturally spoken query so 
that users do not have to speak the search terms exactly (thus supporting 
conversational speech). Machine translation may be utilized to translate search terms 
as well as search results across multiple languages so that the search space may be 
substantially expanded. Automatic summarization techniques may be used to 
summarize the search results if the results are not well organized or are not presented 
in a user-preferred way. Natural language generation and text to speech (TTS) 
techniques may be employed to present the search results back to the user orally in the 
user's native spoken language. The universal voice search concept of the present 
invention, once integrated with an Internet or intranet search engine, becomes a 
powerful tool for people speaking different languages to make use of information 
available on the Internet or an intranet in the most convenient way. This system may 

Serial No. 09/685,419 3 
Examiner: Azad, A. 
Filed: 10/10/2000 
Art Unit: 2654 



promote increased Internet usage among non-English speaking people by making 
search engines or other web sites easier to use. 

Particularly, the present application discloses a method of interfacing to a 
system, as claimed in independent claim 1 and illustrated in Figure 2. At block 100, 
speech input may be received from a user and converted into a digital representation. 
At block 102, the digitized speech may be analyzed to identify the language used by the 
user. At block 104, the speech may be converted into a first text according to the 
identified language by recognizing the user's speech in the speech input data. At block 
106, keywords may be extracted from the first text by parsing the first text. At block 
108, the keywords may be translated into a plurality of automatically selected languages 
other than the identified language. At block 110, the keywords in a plurality of 
languages may be used as search terms for queries to one or more search engines. At 
block 112, the search results in a plurality of languages from the one or more search 
engines may be received and translated into the language used by the user. Next, at 
block 114, the search results may be summarized (if necessary). At block 116, the 
(summarized) search results may be generated in a second text form that represents 
natural language constructs with a prosodic pattern for the user's language. At block 
118, the text may be converted to speech using a text to speech module and rendered 
in an audible manner for the user. See page 10, line 30 to page 1 1, line 14 of the 
specification. 

Additionally, the present application discloses a language independent 
speech based user interface system, as claimed in independent claim 25 and 
shown in Figure 1 . The system comprises a language identifier (22) to receive 
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speech input data from input facility (10) (e.g., voice input (16)), and to identify 
the language spoken by a user (see page 6, line 4 to page 7, line 1 5 of the 
specification). Once the language that the user is speaking is identified, the input 
speech data as well as a language identifier may be passed to the speech 
recognition module (23) which recognizes what words have been said, and 
translates the information into a first text (see page 7, lines 16-28 of the 
specification). Moreover, the system comprises a natural language processing 
module (26) to parse the first text to extract keywords and discard any 
unimportant words (e.g., filler words, repetitions, speech idioms, etc.) within the 
first text (see page 7, line 29 to page 8, line 16 of the specification). 

Once the keywords have been extracted from the text, the keywords may be 
translated by machine translation module (28) into a plurality of supported 
languages. By translating the keywords into multiple languages and using the 
keywords as search terms, the search can be performed across documents in 
different languages, thereby significantly extending the search space used. The 
keywords may be automatically input as search terms in different languages (30) to 
a search engine (32). The search engine searches the Internet or a specified 
intranet and returns the search results in different languages (34). Depending on 
the search results, the results may be in a single language or multiple languages. If 
the search results are in multiple languages, machine translation module (28) may 
be used to translate the search results into the language used by the user. If the 
search results are in a single language that is not the user's language, the results 
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may be translated into the user's language. See page 8, line 17 to page 9, line 8 of 
the specification. 

Furthermore, the system claimed in independent claim 25 comprises an 
automatic summarization module (36) to summarize the search results, if 
necessary; and a natural language generation module (36) to take the summarized 
search results in the user's language and generate naturally spoken forms of the 
results. The results may be modified to conform to readable sentences using a 
selected prosodic pattern so the results sound natural and grammatically correct 
when rendered to the user. See page 9, lines 9-27 of the specification. 

The output of the natural language generation module may be passed to text 
to speech module (20) to convert the text into an audio format and render the 
audio data to the user. Alternatively, the text may be shown on a display 18 in the 
conventional manner. See page 9, line 28 to page 10, line 10 of the specification. 

Regarding independent claim 13, it recites limitations similar to those recited 
in independent claim 1 . In addition to reciting limitations similar to those recited in 
independent claim 25, independent claim 33 recites at least one search engine to 
use the keywords as a search term and to return search results, which is described 
when summarizing claim 25 above. 



VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

Claims 1, 3, 5, 7-10, 12, 13, 15, 17, 19-22, 24-27, 29-30, 33, 35, and 39-42 are 
rejected under 35 U.S.C. 103(a) as being unpatentable over Junqua, et al. (US 
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6,324,512) (hereinafter Junqua), in view of Cohen, et al. (EP 1 014 277) (hereinafter 
Cohen), and further in view of Nosohara (EP 0 838 765) (hereinafter Nosohara). The 
Examiner did not include claim 24 in this rejection, but the Appellant believes that claim 
24 should be included here because it recites limitations similar to those recited in claim 
12. 

Claims 11, 23, and 32 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Junqua in view of Cohen as applied to claims 9, 21, and 29 above, 
and further in view of well-known prior art (MPEP 2144.03). 

VII. ARGUMENT 

A. CLAIMS 1,13, 25, 33 

Independent claims 1,13, 25, and 33 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Junqua in view of Cohen further in view of Nosohara. 

The Examiner has not established a prima facie case of obviousness based 
on a combination of Junqua, Cohen, Nosohara, and the Examiner-asserted well- 
known art. First, to establish a prima facie case of obviousness, "there must be 
some suggestion or motivation, either in the references themselves or in the 
knowledge generally available to one of ordinary skill in the art, to modify the 
reference or to combine the reference teachings." MPEP § 2143. The Examiner 
also recognizes this requirement. See the page 10, first full paragraph of the Final 
Office Action (citing In re Fine 837 F.2d 1071, 5 USPQ.2d 1596 (Fed. Cir. 1988) 
and In re Jones, 958 F.2d 347, 21 USPQ.2d 1941 (Fed. Cir. 1992)). This 
requirement was emphasized several times by the Federal Circuit. For example, in 
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In re Kotzab, 217 F.3d 1365, 55 U.S.P.Q.2d 1313 (Fed. Cir. 2000), the Federal 
Circuit clearly stated, "to establish obviousness based on a combination of the 
elements disclosed in the prior art, there must be some motivation, suggestion or 
teaching of the desirability of making the specific combination that was made by 
the Applicant." (emphasis added). Here, the Appellant respectfully contends that 
Junqua, Cohen, and Nosohara provide no motivation toward a language 
independent voice-based search system as claimed that comprises all of the 
multiple features of language identification, speaker verification, speech 
recognition, natural language processing, automatic summarization, automatic 
translation, speech synthesis, and natural language generation. 

As supported by the earlier submitted declaration from Dr. Yonghong Yan, 
who is an expert in the speech and natural language processing field, technologies 
in this field were previously not sophisticated enough to make possible a language 
independent voice-based search system, as described in the above-identified 
application (Dr. Yonghong Yan's declaration was submitted along with the 
Appellant's response to the Examiner's earlier non-final Office Action dated May 5, 
2004, which resulted in the Final Office Action). Furthermore, prior to the date of 
invention the speed of computing systems was not high enough to make it possible 
for the claimed search system to work in real-time. This explains why in an earlier 
time frame Junqua limited its speech processing application to a very limited 
domain of an electronic program guide (EPG), where the search is conducted in a 
limited size database of specific terms, where no summarization, language 
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identification, translation, or language generation is necessary, a user is expected 
to speak in a predictable way, and responses to a user's requests are very simple. 
Throughout Junqua, there is no suggestion or motivation to expand its teaching to 
a complex application such as the claimed language independent voice-based 
Internet search engine because when Junqua's application was filed in Aug. 1999, 
adequate speech technologies and computing infrastructure were simply not there 
yet. Thus, Junqua could not have suggested (and does not suggest) the present 
claims. The teaching of Cohen is limited to language identification and the 
teaching of Nosohara is limited to text-based language translation. There is no 
motivation or suggestion based on Cohen or Nosohara to expand their limited 
teaching in a single area to a complex application as disclosed in the above- 
identified application, which includes technologies in several complicated areas 
such as speech recognition, speaker verification, speech synthesis, automatic 
summarization, and so on. 

Additionally, one skilled in the art at the time of the invention would not 
have thought to combine these disparate elements as claimed because then exiting 
technologies in speech and natural language processing field were not sophisticated 
enough for complex tasks as disclosed in the above-identified application yet and 
real-time processing of such tasks was previously impossible. Thus, one skilled in 
the art (such as Dr. Yan) would not have thought the present claims were obvious 
at the time of invention. 
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In response to Appellant's above argument that there is no suggestion or 

motivation to combine the references, the Examiner simply states, 

In this case, the motivation or suggestion is found either in the 
reference or in the knowledge generally available to one or ordinary 
skill in the art to combine these references. Motivation or suggest are 
[sic] as follows: 

Therefore, it would have been obvious to one of ordinary skill in 
the art at the time of the invention to automatically summarize the 
search results because one ordinary skill would readily recognize the 
convenience of providing [an] easy understandable summarized search 
results instead of providing whole content of search results. 

Therefore, it would have been obvious to one of ordinary skill in 
the art at the time of the invention to identify a language spoken by 
user as Cohen teaches so that a [sic] language identification is 
achieved for enhancing the speech recognition process from a plurality 
of languages (Cohen, col. 2, lines 15-19). 

Therefore, it would have been obvious to one of ordinary skill in 
the art at the time of the invention to use Nosohara's teaching so that 
optimum search result is obtained from a variety of databases of 
different languages using translation of the keyword and display the 
search results by translating back to original language for user 
convenience (col. 1, lines 8-15). 

See page 10, first full paragraph through page 11, second full paragraph of the 
Final Office Action. 

Here the Examiner's conclusive statements, without providing any support in 
the references themselves or in the knowledge generally available to one of 
ordinary skill in the art, at most show that it would have been obvious to the 
Examiner himself in hindsight after dissecting the elements claimed in the present 
application. According to CAFC case law, "particular factual findings regarding the 
suggestion, teaching, or motivation to combine serve a number of important 
purposes, including: (1) clear explication of the position adopted by the Examiner 
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and the Board; (2) identification of the factual disputes, if any, between the 
applicant and the Board; and (3) facilitation of review on appeal." In re Dembiczak, 
175 F.3d 994, 999-1000 (Fed. Cir. 1999). Here, however, the Examiner failed to 
make any particular findings regarding the suggestion or motivation to combine the 
prior art references. 

In fact, as argued by the Appellant above, Junqua only deals with a limited 
application and there is no suggestion or motivation in Junqua to combine its 
teachings with Cohen, Nosohara, and the Examiner-asserted well-known art. 
Junqua does not need the language identification capability because the application 
was targeted for a specific user using the language the user speaks. Junqua does 
not need the automatic summarization capability because Junqua's search results 
come from an EPG database where all information has already been very well 
organized in a succinct way. Additionally, Junqua does not need the language 
translation capability either because the user searches a database that is in his/her 
language. Furthermore, one of ordinary skill in the art, at the time of the present 
invention, had no motivation or desire to combine these complex technologies 
together because the technologies were not sophisticated yet and computing 
power was not ready for such a combination, as supported by the declaration from 
Dr. Yonghong Yan. 

Because there is no suggestion or motivation, either in the references 
themselves or in the knowledge generally available to one of ordinary skill in the 
art, to modify the references or to combine the reference teachings at the time of 
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the invention, the first basic criterion for a prima facie case of obviousness has not 
been met. Accordingly, any assertion of obviousness based on the cited art and 
the Examiner-asserted "well known" art is in error. Independent claims 1,13, 25, 
and 33 are thus allowable. 

Second, to establish a prima facie case of obviousness, there must also be a 
reasonable expectation of success, which must be found in the prior art references. 
MPEP § 2143. As stated by Dr. Yan in the submitted declaration, technologies in 
speech and natural language processing field were previously not sophisticated 
enough to make possible a language independent voice-based search system, as 
described in the above-identified application. Furthermore, prior to the date of 
invention the speed of computing systems was not high enough to make it possible 
for the claimed search system to work in real-time. Each of Junqua, Cohen, and 
Nosohara is limited to a simple application which then-existing technologies could 
support. No possibility of success can be found, at the time of the invention, in 
either reference for combining them together to come up with a complex search 
system as claimed in the above-identified application. 

In response to the Appellant's above argument that there was no reasonable 
expectation of success of the combination, the Examiner stated in the Final Office 
Action: 

[T]he test for obviousness is not whether the features of a secondary 
reference may be bodily incorporated into the structure of the primary 
reference; nor is it that the claimed invention must be expressly 
suggested in any one or all of the references. Rather, the test is what 
the combined teachings of the references would have suggested to 
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those of ordinary skill in the art (citing In re Keller, 642 F.2d 413, 208 
USPQ 871 (CCPA 1981)). 

See page 12, first full paragraph of the Final Office Action. 

The Examiner seems to miss the point. The quoted statement from In re 
Keller is used to rebut an argument that prior art devices are not physically 
combinable. MPEP 2145(111). However, the Appellant argued that there is no 
reasonable expectation of success of the combination at the time of the invention 
because of the lack of sophistication of technologies and the lack of computing 
power at the time of the invention, as supported by the declaration of Dr. 
Yonghong Yan. According to the MPEP, applicants may present evidence showing 
there was no reasonable expectation of success. MPEP 2143.02. When such 
evidence is provided, the Examiner is required to consider it. "The evidence that 
the combination was not viewed as technically feasible must be considered, for 
conventional wisdom that a combination should not be made is evidence of 
unobviousness." Arkie Lures, Inc. v. Gene Larew Tackle, lnc. f 119 F.3d 953, 958 
(Fed. Cir. 1997). Here the Appellant did submit such evidence, which is the 
declaration of Dr. Yonghong Yan. However, the Examiner failed to consider this 
evidence. Therefore, the Examiner did not rebut the Appellant's argument that 
there was no reasonable expectation of success. 

Because the Examiner has not met at least two basic criteria of a prima facie 
case of obviousness based on a combination of Junqua, Cohen, Nosohara, and the 
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Examiner's own asserted well-known art, the 35 U.S.C. § 103 rejections of claims 

1, 13, 25 and 33 are without foundation. These claims are thus allowable. 

Furthermore, any rejection based on a hindsight-based obviousness analysis 

is expressly forbidden by the case law: 

Our case law makes clear that the best defense against the 
subtle but powerful attraction of a hindsight-based obviousness 
analysis is rigorous application of the requirement for a showing of the 
teaching or motivation to combine prior art references.... Combining 
prior art references without evidence of such a suggestion, teaching, 
or motivation simply takes the inventor's disclosure as a blueprint for 
piecing together the prior art to defeat patentability - the essence of 
hindsight (emphasis added). 

In re Dembiczak, 175 F.3d 994, 50 USPQ2d 1614, 1617 (Fed. Cir. 1999). 

The Examiner appears to be taking the Appellant's disclosure as a blueprint 
for piecing together in hindsight the prior art to attempt to defeat the patentability 
of the present invention. For example, in order to assert that there was suggestion 
or motivation to combine cited prior art references, the Examiner simply asserted 
that combining the automatic summarization element, the language identification 
element, and machine translation element to Junqua's teaching would be 
convenient for a user of Junqua's application (which is actually not true because a 
user of the Junqua application does not need these elements at all). Additionally, 
the Examiner failed to take the timeframe of the invention into account when 
making the rejections based on Junqua, Cohen, and Nosohara. "Measuring a 
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claimed invention against the standard established by section 103 requires the oft- 
difficult but critical step of casting the mind back to the time of invention, to 
consider the thinking of one of ordinary skill in the art, guided only by the prior art 
references and the then-accepted wisdom in the field." Id. at 999. What may 
seem obvious to the Examiner today based on current computing technologies and 
the impermissible use of hindsight was not obvious to one skilled in the art at the 
time of invention, because researchers like Dr. Yan recognized the great difficulties 
involved at the time of the invention. The combination of the cited art does not 
provide any suggestion, either implicit or explicit, that such processing as is 
currently claimed (taking the claim as a whole) could be done, at the time of the 
invention. The cited art does not provide the requisite motivation or suggestion to 
combine the reference teachings. Accordingly, Appellant respectfully requests that 
the 35 U.S.C. 103(a) rejections of independent claims 1,13, 25, and 33 based on 
a combination of Junqua, Cohen, and Nosohara be withdrawn due to the assertion 
of unobviousness by Dr. Yan in the accompanying declaration. 

In response to Appellant's argument that the Examiner's conclusion of 
obviousness is based upon improper hindsight reasoning, the Examiner quoted from 
In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971), "[a]ny judgment 
on obviousness is in a sense necessarily a reconstruction based on hindsight 
reasoning, but so long as it takes into account only knowledge which was within 
the level of ordinary skill in the art at the time the claimed invention was made and 
does not include knowledge gleaned only from applicant's disclosure, such a 
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reconstruction is proper." See page 13, first full paragraph of the Final Office 
Action. However, the Examiner misread In re McLaughlin because it expressly 
requires that the Examiner's judgment of obviousness take into account "only 
knowledge which was within the level of ordinary skill in the art at the time the 
claimed invention was made and does not include knowledge gleaned only from 
applicant's disclosure" (emphases added). Id. Here the declaration of Dr. Yan 
expressly shows that the Examiner's judgment of obviousness could not be 
reached by taking into account only knowledge which was within the level of 
ordinary skill in the art at the time the claimed invention was made because at the 
time of the invention, a person of ordinary skill would not have believed that the 
claimed language-independent search system was possible. Thus, the Examiner's 
judgment of obviousness must have been from impermissible hindsight without any 
finding in the references themselves that suggestion or motivation to combine 
those cited references, either explicit or implicit. 

Because the Examiner did not present any factual findings supporting his 
conclusion that there is suggestion or motivation to combine Junqua, Cohen, Nosohara, 
and the Examiner asserted prior art to come up with the claimed invention in the present 
application, the subject matter claimed in claims 1,13, 25, and 33 is not obvious over 
Junqua in view of Cohen further in view of Nosohara further in view of well-known prior 
art. Thus, a prima facie case of obviousness rejection of claims 1,13, 25, and 33 under 
35 U.S.C. § 103(a) over Junqua in view of Cohen further in view of Nosohara has not 
been made by the Examiner. Therefore, the 35 U.S.C. § 103(a) rejection of claims 1, 
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13, 25, and 33 must be withdrawn and these claims are patentable over Junqua in view 
of Cohen further in view of Nosohara. 

B. CLAIMS , 3, 5, 7-1 2, 15, 17, 19-24, 26-27, 29-30, 32, 35, AND 39-42 

Because independent claims 1, 13, 25, and 33 are patentable, all claims 
dependent therefrom are also patentable (e.g., existing claims 3, 5, 7-12, 15, 17, 
19-24, 26-27, 29-30, 32, 35 and 39-42). 

VIII. CONCLUSION 

Appellant respectfully submits that all the pending claims in this patent 
application are patentable and request that the Board of Patent Appeals and 
Interferences overrule the Examiner and direct allowance of the rejected claims. 

This brief is submitted along with payment of the appeal fee for one other than a 
small entity as specified in 37 C.F.R. § 1.17(c). Please charge any shortages and credit 
any overcharges to Deposit Account No. 02-2666. 

Respectfully submitted, 

s/Steven P. Skabrat/ 
Reg. No. 36,279 
Intel Corporation 
Attorney for Appellant 
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1 IX. APPENDIX A: CLAIMS ON APPEAL 

2 

3 1 . A method of interfacing to a system comprising: 

4 receiving speech input data from a user; 

5 identifying a language spoken by the user from the speech input data; 

6 converting the speech input data into a first text in the identified language by 

7 recognizing the user's speech in the speech input data based at least in part on the 

8 language identifier; 

9 parsing the first text to extract keywords; 

10 automatically translating the keywords into a plurality of automatically 

1 1 selected languages other than the identified language; 

12 using the translated keywords as a command to an application; 

13 receiving results to the command; 

14 automatically summarizing the results; 

15 converting the summarized results into a second text with a prosodic pattern 

16 according to the language spoken by the user; and 

17 rendering the second text for perception by the user. 
1 

1 2. (Cancelled) 
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1 3. The method of claim 1, wherein rendering comprises converting the 

2 second text into speech and rendering the speech to the user. 

1 

1 4. (Cancelled) 

1 

1 5. The method of claim 1, further comprising using the keywords as a 

2 search query to at least one search engine, wherein the results comprise search 

3 results from the at least one search engine operating on the search query. 

1 

1 6. (Cancelled) 

1 

1 7. The method of claim 1 , further comprising automatically translating the 

2 keywords into a plurality of automatically selected languages other than the 

3 identified language and using the translated keywords as a search query to at least 

4 one search engine in multiple languages, wherein the results comprise search 

5 results in multiple languages from the at least one search engine operating on the 

6 search query. 



1 
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1 8. The method of claim 7, further comprising automatically translating 

2 search results in languages other than the language spoken by the user into the 

3 language spoken by the user. 

1 



1 9. The method of claim 1, wherein the application comprises a web 

2 browser. 

1 

1 10. The method of claim 9, wherein the web browser interfaces with at 

2 least one search engine and the command comprises a search query. 

1 

1 11. The method of claim 9, wherein the web browser interfaces with a 

2 shopping web site and the command comprises at least one of a purchase order 

3 and a request for product information. 

1 

1 12. The method of claim 1, wherein the speech comprises conversational 

2 speech. 

1 

1 13. An article comprising: a storage medium having a plurality of machine 

2 readable instructions, wherein when the instructions are executed by a processor, 
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3 the instructions provide for interfacing to a system by receiving speech input data 

4 from a user, identifying a language spoken by the user from the speech input data, 

5 converting the speech input data into a first text in the identified language by 

6 recognizing the user's speech in the speech input data based at least in part on the 

7 language identifier, parsing the first text to extract keywords, automatically 

8 translating the keywords into a plurality of automatically selected languages other 

9 than the identified language, using the translated keywords as a command to an 

10 application, receiving results to the command, automatically summarizing the 

1 1 results, converting the summarized results into a second text a prosodic pattern 

12 according to the language spoken by the user, and rendering the second text for 

1 3 perception by the user. 

1 

1 14. (Cancelled) 

1 

1 15. The article of claim 13, wherein instructions for rendering comprise 

2 instructions for converting the second text into speech and rendering the speech to 

3 the user. 

1 

1 16. (Cancelled) 

1 
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1 17. The article of claim 13, further comprising instructions for using the 

2 keywords as a search query to at least one search engine, wherein the results 

3 comprise search results from the at least one search engine operating on the search 

4 query. 

1 

1 18. (Cancelled) 

1 

1 19. The article of claim 13, further comprising instructions for automatically 

2 translating the keywords into a plurality of automatically selected languages other 

3 than the identified language and using the translated keywords as a search query to 

4 at least one search engine in multiple languages, wherein the results comprise 

5 search results in multiple languages from the at least one search engine operating 

6 on the search query. 

1 

1 20. The article of claim 19, further comprising instructions for 

2 automatically translating search results in languages other than the language 

3 spoken by the user into the language spoken by the user. 

1 

1 21 . The article of claim 13, wherein the application comprises a web 

2 browser. 
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1 22. The article of claim 21, wherein the web browser interfaces with at 

2 least one search engine and the command comprises a search query. 

1 

1 23. The article of claim 21, wherein the web browser interfaces with a 

2 shopping web site and the command comprises at least one of a purchase order 

3 and a request for product information. 

1 

1 24. The article of claim 13, wherein the speech comprises conversational 

2 speech. 

1 

1 25. A language independent speech based user interface system 

2 comprising: 

3 a language identifier to receive speech input data from a user and to identify 

4 the language spoken by the user; 

5 at least one speech recognizer to receive the speech input data and the 

6 language identifier and to convert the speech input data into a first text based at 

7 least in part on the language identifier; 
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8 at least one natural language processing module to parse the first text to 

9 extract keywords; 

10 at least one summarization module to automatically summarize the search 

1 1 results from at least one search engine operating on the search query using the 

12 extracted keywords; 

13 at least one language translator to automatically translate the keywords into 

14 a plurality of automatically selected languages other than the identified language for 
1 5 use as a command to an application, and to translated results to the command in 

1 6 languages other than a language spoken by the user to the language spoken by the 

1 7 user; and 

18 at least one natural language generator to convert the summarized results 

1 9 into a second text with a prosodic pattern according to the language spoken by the 

20 user. 

1 

1 26. The system of claim 25, further comprising at least one text to speech 

2 module to render the second text audibly to the user. 

1 

1 27. The system of claim 25, further comprising at least one language 

2 translator to automatically translate the keywords into a plurality of automatically 

3 selected languages for use as a search query, and to automatically translate the 
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4 search results in languages other than the language spoken by the user into the 

5 language spoken by the user prior to summarizing the translated results and 

6 converting the summarized results into the second text in a natural language 

7 format. 

1 

1 28. (Cancelled) 

1 

1 29. The system of claim 25, wherein the system is coupled to a web 

2 browser. 

1 

1 30. The system of claim 29, wherein the web browser interfaces with at 

2 least one search engine, the keyword comprises a search query, and the second 

3 text comprises search results from the at least one search engine. 

1 

1 31. (Cancelled) 

1 

1 32. The system of claim 29, wherein the web browser interfaces with a 

2 shopping web site and the command comprises at least one of a purchase order 

3 and a request for product information. 
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1 33. A language independent speech based search system comprising: 

2 a language identifier to receive speech input data from a user and to identify 

3 the language spoken by the user; 

4 at least one speech recognizer to receive the speech input data and the 

5 language identifier and to convert the speech input data into a first text based at 

6 least in part on the language identifier; 

7 at least one natural language processing module to parse the first text to 

8 extract keywords; 

9 at least one search engine to use the keywords as a search term and to 

10 return search results; 

1 1 at least one language translator to automatically translate the keyword into a 

12 plurality of automatically selected languages prior to input to the at least one 

13 search engine to search across multiple languages, and to automatically translate 

14 search results in languages other than the language spoken by the user into the 
1 5 language spoken by the user; 

16 at least one automatic summarization module to automatically summarize the 

17 translated search results; 

18 at least one natural language generator to convert the summarized results 

19 into a second text with a prosodic pattern according to the language spoken by the 

20 user. 
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34. (Cancelled) 



35. The system of claim 33, further comprising at least one text to speech 
2 module to render the second text audibly to the user. 



36-38. (Cancelled) 



39. The method of claim 1 , wherein the prosodic pattern is capable of 
2 making the second text sound natural and grammatically correct. 



40. The article of claim 13, wherein the prosodic pattern makes the second 
2 text sound natural and grammatically correct. 



41 . The system of claim 25, wherein the prosodic pattern makes the 
2 second text sound natural and grammatically correct. 

1 
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42. The system of claim 33, wherein the prosodic pattern makes the second text 
sound natural and grammatically correct. 
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